The present invention relates generally to data processing, and more particularly, to managing batch operations in an enterprise data integration platform environment.
Data warehouse systems provide businesses with useful and meaningful data that can be used to support decision making with respect to various aspects of a business enterprise. Data from various enterprise databases may be extracted and combined within a data warehouse supported by the data warehouse system using, e.g., an Extract, Transfer, and Load (ETL) tool. The ETL tool typically includes a scheduling component that ensures operations on the data are performed in a set order.
These ETL processes may be implemented in conjunction with other applications (e.g., business applications of the enterprise). A process schedule is devised and read by the scheduling component, which accesses one or more of the applications based upon the information in the schedule to perform the operations in the process schedule.
Scheduling tasks performed by existing ETL tools offer limited flexibility in terms of error logging, implementing batch dependencies, and messaging, to name a few. This is because many of these features are typically hard coded into the ETL tools, thereby offering little in the way of customization.
What is needed, therefore, is a way to provide increased flexibility and customization in batch scheduling processes.